Embodied Musical Interaction

Music is a natural partner to human-computer interaction, offering tasks and use cases for novel forms of interaction. The richness of the relationship between a performer and their instrument in expressive musical performance can provide valuable insight to human-computer interaction (HCI) researchers interested in applying these forms of deep interaction to other fields. Despite the longstanding connection between music and HCI, it is not an automatic one, and its history arguably points to as many differences as it does overlaps. Music research and HCI research both encompass broad issues, and utilize a wide range of methods. In this chapter I discuss how the concept of embodied interaction can be one way to think about music interaction. I propose how the three “paradigms” of HCI and three design accounts from the interaction design literature can serve as a lens through which to consider types of music HCI. I use this conceptual framework to discuss three different musical projects—Haptic Wave, Form Follows Sound, and BioMuse

Timbre from Sound Synthesis and High-level Control Perspectives

International audienceExploring the many surprising facets of timbre through sound manipulations has been a common practice among composers and instrument makers of all times. The digital era radically changed the approach to sounds thanks to the unlimited possibilities offered by computers that made it possible to investigate sounds without physical constraints. In this chapter we describe investigations on timbre based on the analysis by synthesis approach that consists in using digital synthesis algorithms to reproduce sounds and further modify the parameters of the algorithms to investigate their perceptual relevance. In the first part of the chapter timbre is investigated in a musical context. An examination of the sound quality of different wood species for xylophone making is first presented. Then the influence of instrumental control on timbre is described in the case of clarinet and cello performances. In the second part of the chapter, we mainly focus on the identification of sound morphologies, so called invariant sound structures responsible for the evocations induced by environmental sounds by relating basic signal descriptors and timbre descriptors to evocations in the case of car door noises, motor noises, solid objects, and their interactions

Accelerating Multi-Channel Filtering of Audio Signal on ARM Processors

Tablets and smart phones are nowadays equipped with low-power processor architectures such as the ARMv7 and the ARMv8 series. These processors integrate powerful SIMD units to exploit the intrinsic data-parallelism of most media and signal processing applications. In audio signal processing, there exist multiple problems that require filtering operations such as equalizations or signal synthesizers, among others. Most of these applications can be efficiently executed today on mobile devices by leveraging the processor SIMD unit. In this paper, we target the implementation of multi-channel filtering of audio signals on ARM architectures. To this end, we consider two common audio filter structures: FIR and IIR. The latter is analyzed in two different forms: direct form I and parallel form. Our results show that the SIMD-accelerated implementation increases the processing speed by a factor of 4 × with respect to the original code, and our hand-tuned SIMD implementation outperforms the auto- vectorized code by a factor of 2× . These results allow us to deal in real time with multi-channel systems composed of 260 FIR filters with 256 coefficients, or 125 IIR filters with 256 coefficients, of INT16 data type.The researchers from Universitat Jaume I are supported by the CICYT projects TIN2014-53495-R and TIN2011-23283 of the Ministerio de Economia y Competitividad and FEDER. The authors from the Universitat Politecnica de Valencia are supported by projects TEC2015-67387-C4-1-R and PROMETEOII/2014/003. This work was also supported from the European Union FEDER (CAPAP-H5 network TIN2014-53522-REDT).Belloch Rodríguez, JA.; Alventosa, FJ.; Alonso-Jordá, P.; Quintana Ortí, ES.; Vidal Maciá, AM. (2017). Accelerating Multi-Channel Filtering of Audio Signal on ARM Processors. Journal of Supercomputing. 73(1):203-214. https://doi.org/10.1007/s11227-016-1689-8S203214731ARM NEON. http://www.arm.com/ . Accessed 23 Feb 2015Rämo J, Välimäki V, Bank B (2014) High-precision parallel graphic equalizer. IEEE Trans Audio Speech Lang Process 22:1894–1904Mathews MV, Miller JE, Moore FR, Pierce JR, Risset JC (1969) The technology of computer music. MIT Press, Cambridge, MassRisset JC (1985) Computer music experiments 185. Comput Music J 22:11–18Puckette M (2007) The theory and technique of electronic music, World Scientific Publishing ISBN-13: 978–9812700773Savioja L, Välimäki V, Smith JO (2011) Audio signal processing using graphics processing units. J Audio Eng Soc 59:3–19Belloch JA, Bank B, Savioja L, Gonzalez A, Välimäki V (2014) Multi-channel IIR filtering of audio signals using a GPU. In: Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP-14), pp 6692–6696Belloch JA, Gonzalez A, Martnez-Zaldívar FJ, Vidal AM (2013) Multichannel massive audio processing for a generalized crosstalk cancellation and equalization application using GPUs. Integr Comput Aided Eng 20:169–182Algazi V, Duda R (2011) Headphone-based spatial sound. IEEE Signal Process Mag 28:33–42Belloch JA, Ferrer M, Gonzalez A, Martinez-Zaldívar FJ, Vidal AM (2013) Headphone-based virtual spatialization of sound with a GPU accelerator. J Audio Eng Soc 61:546–556Huang Y, Chen J, Benesty J (2011) Immerse audio schemes. IEEE Signal Process Mag 28:20–32Oppenheim AV, Willsky AS, Hamid S (1997) Signals and systems, processing series, 2nd edn. Prentice Hall, Upper Saddle RiverBank B (2008) Perceptually motivated audio equalization using fixed-pole parallel second-order filters. IEEE Signal Process Lett 15:477–480Mitra G, Johnston B, Rendell AP, McCreath E, Zhou J (2013) Use of SIMD vector operations to accelerate application code performance on low-powered ARM and Intel Platforms. In: IEEE 27th International Parallel and Distributed Processing Symposium Workshops PhD Forum (IPDPSW), pp 1107–1116Welch E, Patru D, Saber E, Bengtson K (2012) A study of the use of SIMD instructions for two image processing algorithms. Western New York Image Processing Workshop (WNYIPW), pp 21–24Wang R, Wan J, Wang W, Wang Z, Dong S, Gao W (2013) High definition IEEE AVS decoder on ARM NEON platform. In: 20th IEEE International Conference on Image Processing (ICIP), pp 1524–1527Holgersson SB (2012) Optimising IIR filters using ARM NEON, Master Thesis of University of DenmarkRabiner LR, Gold B (1975) Theory and application of digital signal processing. Prentice-Hall, Englewood CliffsARM NEON intrinsics. http://gcc.gnu.org/onlinedocs/gcc-4.4.1/gcc/ARM-NEON-Intrinsics.html . Accessed 12 July 2015ARM NEON auto-vectorization. http://gcc.gnu.org/onlinedocs/gcc/ARM-Options.html . Accessed 22 July 201

Prospective view on sound synthesis BCI control in light of two paradigms of cognitive neuroscience

International audienceDifferent trends and perspectives on sound synthesis control issues within a cognitive neuroscience framework are addressed in this article. Two approaches for sound synthesis based on the modelling of physical sources and on the modelling of perceptual effects involving the identification of invariant sound morphologies (linked to sound semiotics) are exposed. Depending on the chosen approach, we as- sume that the resulting synthesis models can fall under either one of the theoretical frameworks inspired by the representational-computational or enactive paradigms. In particular, a change of viewpoint on the epistemological position of the end user from a third to a first person, inherently involves different conceptualizations of the interaction between the listener and the sounding object. This differentiation also influences the design of the control strategy enabling an expert or an intuitive sound manipulation. Finally, as a perspective to this survey, Explicit and Implicit Brain Control Interfaces (BCI) are described with respect to the previous theoreti- cal frameworks, and a semiotic-based BCI aiming at increasing the intuitiveness of synthesis control processes is envisaged. These interfaces may open for new appli- cations adapted to either handicapped or healthy subjects